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Abstract 


Life  Cycle  Cost  (LCC)  assessments  are  of  interest  during  the  design  phase 
for  new  systems.  These  often  involve  costs  that  must  be  estimated  from  a  variety  of 
different  sub-models,  including  cost  models  constructed  from  historical  data,  forecast 
models  that  attempt  to  predict  future  economic  conditions,  and  economy-of-scale 
models  that  impact  production  schedules,  and  more.  When  these  disparate  models 
are  put  together  to  obtain  an  overall  cost  model,  many  of  these  individual  sources  of 
uncertainty  end  up  being  aggregated  or  ignored.  Consequently,  the  cost  estimates 
may  not  provide  program  managers  with  appropriate  assessments  of  the  risk  and 
overall  variability  of  the  new  systems.  We  propose  a  structured  approach  for 
obtaining  robust  LCC  estimates  by  taking  into  account  a  broad  set  of  environmental 
noise  conditions.  This  will  enable  program  managers  to  better  understand  the 
uncertainty  in  their  overall  estimates,  and  to  identify  any  decision  factor 
combinations  that  result  in  both  low  costs  and  low  cost  variability.  This  may  provide 
guidance  on  which  of  the  many  potential  uncertainty  sources  require  close 
monitoring,  and  which  can  safely  be  disregarded.  We  illustrate  this  approach  with  a 
model  the  USMC  is  evaluating  for  use  in  cost/benefit  analysis  of  alternative  energy 
systems. 
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A  Robust  Design  Approach  to  Cost 
Estimation:  Soiar  Energy  for  Marine  Corps 
Expeditionary  Operations 


Introduction 

Taguchi  made  a  major  contribution  to  the  quality  engineering  literature 
through  his  vision  of  building  quality  into  the  product  at  the  manufacturing  design 
level  (Taguchi,  1986,  1987;  Taguchi  &  Wu,  1980).  He  recognized  that  a  careful 
choice  of  settings  for  design  decision  factors  (or  parameters)  could  lead  to  products 
that  were  insensitive  to  uncertainties  in  the  manufacturing  and  customer 
environment.  The  uncertainties,  or  noise  factors  were  uncontrollable  (or  controllable 
only  at  great  expense)  in  the  real  world,  and  could  either  be  internal  (based  on 
endogenous  characteristics  of  the  system)  or  external  (resulting  from  exogenous 
effects).  However,  by  manipulating  both  the  decision  factors  and  the  noise  factors 
during  laboratory  experiments,  a  statistical  design-of-experiments  approach  could  be 
used  to  systematically  seek  improved  product  designs. 

In  the  parameter  design  (or  robust  design)  stage,  the  analyst  conducts  a 
designed  experiment  to  gather  information  about  the  expected  system  performance 
across  the  noise  space.  The  ideal  system  configuration  is  one  that  results  in  a  mean 
performance  equal  to  the  target  r,  and  a  performance  variability  of  zero.  Since  in 
practice  the  ideal  is  unattainable,  a  loss  function  serves  to  trade  off  average 
deviation  from  the  target  with  consistency  of  the  output.  The  “best”  system  is  thus 
often  not  that  associated  with  the  best  mean  performance.  For  example,  Quinlan 
(1985)  described  how  a  redesigned  speedometer  cable  resulted  in  an  over  ten-fold 
savings  in  expected  warranty  claims  because  of  reductions  in  the  mean  and 
variance  of  cable  shrinkage.  Sanchez  et  al.  (1 998)  examine  a  job  shop  where  loss 
is  a  function  of  the  time  jobs  are  in  the  system.  They  found  the  optimal  shop  floor 
layout  and  control  resulted  in  over  35%  savings  when  compared  to  the  configuration 
yielding  the  lowest  mean.  Risk  and  performance  trade-offs  are  routinely  considered 
in  other  fields,  such  as  financial  portfolio  management,  although  they  have  not  yet 
benefitted  from  the  formal  application  of  the  robust  design  approach. 

The  final  stage  in  Taguchi's  framework  is  called  tolerance  design  (Taguchi 
1986,  1987;  see  also  D’Errico  &  Zaino,  1988.)  At  this  stage,  the  levels  for  the 
decision  factors  have  already  been  chosen  to  achieve  a  good  product  design.  The 
analyst  conducts  a  further  statistical  experiment  to  evaluate  the  overall  system 
performance  and  attribute  variation  in  the  response  to  variation  in  the  noise  factors. 
Any  decision  factors  that  cannot  be  precisely  controlled  in  the  manufacturing 
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environment,  but  are  randomly  distributed  about  their  chosen  values,  are  also 
considered  sources  of  noise.  By  incorporating  the  loss  function,  the  tolerance 
design  results  can  suggest  several  ways  for  further  improving  the  product  design. 
They  indicate  whether  or  not  upgrades  in  the  consistency  of  component  parts  are 
cost-effective,  or  whether  any  specifications  could  be  relaxed  without  adversely 
affecting  product  quality.  In  addition,  by  quantifying  the  impact  of  the  noise  factors 
on  the  expected  loss,  they  highlight  which  (if  any)  sources  of  noise  should  be 
targeted  for  improvement  in  the  future. 

In  this  paper,  we  focus  on  using  designed  experiments  to  evaluate  the  life 
cycle  cost  of  a  particular  system,  where  the  noise  factors  represent  uncertainties  in 
future  component  costs.  In  manufacturing  applications,  tolerance  design 
experiments  are  typically  set  up  to  detect  only  main  effects,  either  using  Taguchi's 
orthogonal  arrays  or  other  classic  orthogonal  designs.  Analysts  have,  by 
construction,  guaranteed  that  the  noise  factor  settings  are  uncorrelated  during  the 
experiment.  However,  it  is  still  possible  for  some  of  these  factors  to  be  correlated — 
perhaps  strongly — in  the  real  world  setting  under  normal  operating  conditions.  If  so, 
this  raises  several  questions:  What  effect  does  correlation  have  on  the  tolerance 
design  results?  How  can  data  obtained  from  an  orthogonal  experiment  be  used  for 
noise  factor  assessment  when  two  or  more  of  these  factors  are  correlated?  Are 
there  better  alternatives  for  the  experimental  design  if  correlation  is  known  to  exist  a 
priorH 

Our  goal  in  this  paper  is  to  show  that  it  is  possible  to  construct  robust  cost  or 
performance  estimates  for  situations  where  strong  pairwise  correlations  are  present 
among  the  noise  factors.  We  motivate  the  need  for  robust  cost  estimation  using  an 
example  of  current  interest  to  the  Marine  Corps  Expeditionary  Energy  Office.  We 
then  review  a  response-surface  modeling  approach  for  tolerance  design  (Mining  & 
Myers,  1990;  Myers  et  al.,  1991 ;  Bamberg  et  al.  1991),  describe  modifications  in  the 
analysis  to  account  for  such  correlation,  and  present  a  class  of  elliptical  designs  that 
can  be  used  when  orthogonal  designs  over  the  entire  range  of  interest  are 
inappropriate. 

Motivating  Example;  Marine  Corps  Expeditionary 
Energy 

Our  motivating  example  is  drawn  from  Morse  (2014),  who  explores  in  great 
detail  the  performance  of  a  model  that  can  be  used  to  predict  the  energy  needs  for 
Marine  Corps  expeditionary  operations.  In  this  paper,  we  consider  a  single  use-case 
from  Morse's  study  to  illustrate  the  need  for  robust  cost  estimates,  as  well  as  how 
appropriate  experimental  designs  can  be  used  to  provide  more  insight  to  program 
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managers  and  planners  about  the  projected  operational  cost/benefit  of  investments 
in  alternative  energy. 

Marine  Corps  Expeditionary  Energy  Initiatives 

General  Amos  published  the  Marine  Corps  Expeditionary  Energy  Strategy  on 
March  21 ,2011.  This  energy  strategy  sets  an  objective  of  making  50%  of  Marine 
Corps  bases  “net-zero  energy  consumers”  by  2020  (ALMAR01 1  /1 1 , 201 1 ).  In 
201 1 ,  fuel  consumption  in  Afghanistan  was  estimated  at  8  gallons  per  Marine  per 
day  (Hersterman,  2011).  In  the  Expeditionary  Energy  Strategy,  General  Amos 
focused  on  the  deployed  environment  and  called  for  a  50  percent  reduction  in  fuel 
consumption  per  Marine  down  to  4  gallons  per  Marine  per  day  (Amos,  n.d.).  The 
Marine  Corps’  reduction  in  fuel  consumption  will  result  not  only  in  increased 
maneuverability  and  sustainability,  but  also  in  reduced  fuel-supply  related  casualties. 

To  accomplish  this  reduction  in  fuel  consumption,  the  Marine  Corps  first 
focused  on  reducing  the  fuel  consumption  of  forward  operating  bases  (FOB).  The 
Marine  Energy  Assessment  team  identified  that  32  percent  of  fuel  consumed  by  the 
Marine  Expeditionary  Brigade  (MEB)  deployed  to  Afghanistan  in  2009  was  used  for 
electric  power  generation  (Schwartz,  Blakeley,  &  O'Rourke,  2012). 

Through  the  efforts  of  the  Expeditionary  Energy  Office,  the  Office  of  Naval 
Research  (ONR),  and  their  interaction  with  industry,  the  Marine  Corps  has 
developed  robust,  renewable  technologies  to  meet  the  energy  objectives  laid  out  in 
the  Commandant's  energy  strategy.  Through  the  EEO  the  Marine  Corps  has 
investigated  energy  solutions  in  such  areas  as  photovoltaic  arrays  for  power 
harvesting,  light  emitting  diodes  (LED)  for  decreased  energy  consumption,  and 
improved  battery  and  smart  power  controller  technologies  for  better  energy 
management. 

Two  successful  renewable  energy  assets  developed  by  the  Marine  Corps  are 
the  Ground  Renewable  Expeditionary  Energy  System  (GREENS)  and  the  Solar 
Portable  Alternative  Communications  Energy  System  (SPACES).  GREENS  is  a 
power  generation  and  conversion  system  that  allows  Marines  to  power  systems  with 
solar  energy.  Each  GREENS  is  comprised  of  eight  photovoltaic  array  panels,  four 
high-energy  lithium  batteries  (HELB),  and  a  central  controller.  Figure  1  shows  how 
all  components  of  a  GREENS  are  combined  to  harvest  solar  power,  receive  AC  and 
DC  power,  employ  the  FIELB  batteries,  and  to  provide  DC  output  for  a  load. 
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DC  Input  Power  AC  Input  Power 
Source  Source 


Figure  1.  GREENS  employed  with  all  photovoltaic  arrays  and  High  Energy 

Lithium  Batteries 

(from  USMC  Warfighting  Laboratory,  2012) 

The  SPACES  system  is  much  smaller,  lighter,  and  less  powerful  than 
GREENS.  Powered  by  two  (12)  volt  folding  portable  solar  panels,  a  single  SPACES 
is  designed  to  recharge  two  BB-2590  lithium  batteries  or  power  battery  operated 
devices  such  as  laptops  or  radios;  SPACES  can  also  be  powered  from  batteries  or 
running  vehicles  if  solar  power  is  unavailable  (United  States  Marine  Corps  Marine 
Corps  Warfighting  Laboratory,  2012). 

While  a  single  GREENS  can  provide  300  watts  of  power  for  24  hours, 
GREENS  systems  are  modular  and  can  be  combined  into  as  large  as  a  five 
GREENS  system,  providing  for  peak  power  requirements  of  up  to  1 000  Watts  (1  kW) 
The  GREENS'  modular  design  also  allows  the  user  to  employ  fewer  than  the  full  8 
solar  panels  and  fewer  than  the  full  four  HELB  batteries  if  desired,  to  configure  the 
GREENS  to  meet  smaller  load  requirements.  The  first  GREENS  unit  was  tested  in 
July  2009.  The  CMC  approved  accelerated  fielding  of  GREENS  and  SPACES  in 
April  201 1 ,  and  fielding  began  in  early  2012  (United  States  Marine  Corps  Center  for 
Lessons  Learned,  2012). 

There  are  two  general  approaches  for  employing  renewable  energy 
technologies  to  reduce  fuel  consumption.  The  first  approach  decreases  the  peak 
levels  of  a  load  by  removing  components  of  that  load  and  powering  them  separately 
with  renewable  energy  technology  such  as  PV  arrays,  batteries,  and  wind  turbines. 
To  ensure  reliable  availability  of  power,  this  will  usually  also  include  using  a 
conventional  generator  as  a  backup  to  supply  power  in  times  of  poor  weather.  The 
Marine  Corps  currently  employs  the  GREENS  in  this  way.  The  second  approach 
employs  the  renewable  energy  technologies  to  decrease  the  operating  time  of  any 
AC  generator(s)  covering  the  load.  This  combines  the  generator  and  renewable 
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energy  source  into  a  hybrid  system  that  passes  the  load  between  renewable  power 
sources  and  diesel  generators  depending  on  load  size  and  renewable  power 
production.  In  many  situations,  this  method  of  employing  a  renewable  energy 
technology  can  do  more  to  decrease  fuel  consumption  and  allow  for  higher 
generator  operating  efficiency  than  the  first  technique  described.  The  Marine  Corps 
began  its  pursuit  of  hybrid  systems  capable  of  powering  3kW  to  300kW  micro-grids 
(United  States  Marine  Corps  Expeditionary  Energy  Office,  2013).  In  this  paper,  we 
focus  on  using  GREENS  in  off-the-grid  capacities. 


Equipment 

Average  Hourly 

Power  Required  (W) 

Peak  Power 
Required  (W) 

GBOSS  Heavy  (w/2  40"  LCDs) 

961 

800 

VRC-110  w/Blue  Force  Tracker 

165 

440 

PRC-150 

57 

375 

Coffee  Pot 

45 

975 

Figure  2.  Examples  of  common  Marine  Corps  equipment  the  GREENS 
system  can  power,  with  associated  power  requirements 

(from  MCWLGuide) 

The  Hybrid  Optimization  of  Multiple  Energy  Resources 
(HOMER) 

Beyond  the  renewable  technologies  and  hybrid  systems  themselves,  another 
tool  the  Marine  Corps  has  pursued  as  part  of  its  energy  strategy  is  the  Hybrid 
Optimization  of  Multiple  Energy  Resources  (HOMER)  model  (HOMER  energy,  n.d.). 
HOMER  is  a  deterministic,  time-step  model  of  micropower  systems.  It  can  model 
both  conventional  systems  such  as  diesel-powered  generators,  and  renewable 
power  sources  such  as  photovoltaic  arrays  and  wind  turbines.  The  HOMER  model 
has  been  utilized  for  years  by  organizations  and  companies  across  the  globe,  to 
assist  them  in  their  cost  effective  employment  of  renewable  energy  sources  and 
appropriately  sized  generators  for  the  long  term  powering  of  facilities.  There  is 
potential  for  HOMER  to  serve  as  a  useful  Marine  Corps  Modeling  and  Simulation 
tool  for  planning  efficient  and  effective  expeditionary  micropower  systems,  by 
helping  Marine  Corps  power  planners  and  logisticians  anticipate  the  fuel 
consumption  savings  to  be  gleaned  from  employing  hybrid  systems  to  support  a 
given  load  requirement  for  a  particular  location  and  time  frame.  To  be  useful  for 
modeling  expeditionary  microgrids,  HOMER  must  be  robust  across  a  wide  range  of 
locations,  microgrids,  compositions,  and  load  profiles. 

The  U.S.  National  Renewable  Energy  Laboratory  (NREL)  developed  HOMER 
in  1 993  as  a  part  of  its  Village  Power  Program.  The  model  was  designed  to  assist 
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users  in  designing  micropower  systems  and  modeling  their  performance  under 
specific  conditions.  Although  originally  built  only  for  consideration  of  off-the-grid 
micropower  systems,  the  model  was  updated  in  2000  to  accommodate  the  modeling 
of  grid-tied  systems  as  well.  The  HOMER  application  simulates  and  measures  the 
performance  of  each  power  generation  asset  available,  both  individually  and  in 
hybrid  configurations.  In  this  way,  the  model  is  used  to  identify  the  most  cost- 
efficient  micropower  system  design  for  meeting  long  term  electric  load  requirements. 

In  order  for  the  HOMER  model  to  simulate  the  performance  of  the  solar 
panels,  the  user  must  load  the  solar  resource  and  temperature  profile  into  the  model. 
For  years,  the  solar  industry  has  relied  upon  NREL's  Typical  Meteorological  Year 
(TMY)  data  set  for  modeling  the  performance  of  solar  power  systems  (Williams  & 
Kerrigan,  201 2).  A  TMY  is  a  statistically  derived  profile  of  a  typical  meteorological 
year  for  certain  locations  across  the  United  States.  NREL's  TMY  data  consists  of 
hourly  measurements  of  solar  irradiance  and  certain  meteorological  elements  and 
has  been  updated  three  times  since  its  establishment  in  1978  by  Sandia  National 
Laboratories  (Wilcox  &  William,  2008).  The  current  TMY,  TMY3,  was  created  with 
measurements  acquired  from  1976  to  2005.  TMY3  data  can  be  imported  into 
HOMER  for  given  localities. 

Previous  HOMER  Assessment 

Marine  Corps  expeditionary  operations  involve  relatively  small  units 
conducting  fast  paced  and  short  duration  operations.  The  units  depend  on  robust, 
space  efficient,  and  weight  efficient  sources  of  power.  Logistics  planners  and 
commanders  must  be  able  to  rely  on  their  energy  source,  and  variability  in  solar 
irradiance  is  one  element  that  brings  that  reliability  into  question.  Solar  irradiance 
variance  can  be  caused  by  multiple  factors,  including  varying  cloud  cover  and 
pollution. 

In  2010,  Major  Brandon  Newell  conducted  an  experiment  where  he  assessed 
the  capabilities  of  the  HOMER  model  in  forecasting  the  power  output  of  a  solar  panel 
at  the  Naval  Postgraduate  School  (Newell,  2010).  This  initial  research  found  that, 
when  employing  monthly  temperatures  averages  and  synthetic  solar  irradiance 
profiles  built  using  the  algorithm  of  Graham  &  Hollands  (1990)  for  model  inputs,  the 
HOMER  model  performed  with  an  unacceptable  degree  of  fidelity,  with  HOMER 
estimating  “an  energy  production  level  that  was  over  25  percent  higher  than  the 
actual  measured  energy"  (Newell,  2010).  Major  Newell  attributed  this  overestimate 
of  power  production  to  three  factors:  a  disparity  between  measured  and  anticipated 
surface  solar  irradiance,  a  disparity  between  measured  and  anticipated  ambient 
temperature,  and  an  underestimate  of  the  de-rating  value  for  the  solar  panels. 
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Major  Newell's  2010  thesis  research  demonstrated  that  solar  irradiance 
variance  from  the  anticipated  values  will  result  in  inaccurate  power  production 
predictions  by  the  HOMER  model.  He  also  demonstrated  that  the  percentage  of 
solar  irradiance  disparity  does  not  account  for  the  percentage  of  inaccuracy  created 
in  predicted  power  output.  While  the  anticipated  solar  irradiance  profile 
overestimated  actual  solar  irradiance  by  9.35%,  this  did  not  equate  to  the  model 
overestimating  PV  power  production  by  9.35% — instead,  the  HOMER  model 
overestimated  the  PV  power  production  by  27%.  As  Newell  (2010)  pointed  out,  this 
is  “due  to  the  nature  of  the  comparison,  which  only  compared  the  total  kW  per  m 
squared  for  the  month  and  disregarded  when  the  disparities  occurred.”  The  amount 
of  solar  irradiance  variability  that  occurs  for  a  given  scenario  does  not  necessarily 
equate  to  an  equivalent  amount  of  PV  power  production  variability,  but  there  is  a 
direct  relationship  between  solar  irradiance  and  PV  production  variability  that  must 
be  identified. 

The  27%  overestimate  in  energy  production,  identified  in  Newell's  thesis  work, 
was  predicted  using  solar  irradiance  and  temperature  profiles.  Solar  irradiance 
profiles  were  created  using  average  monthly  solar  irradiance  values  acquired  from 
NASA  Surface  Solar  Energy  Data  Set  and  employment  of  the  Graham-Hollands 
algorithm  to  create  a  synthetic  profile  based  on  the  average  monthly  solar  irradiance 
values.  Temperature  profiles  consisted  of  monthly  temperature  averages  for  the 
location.  To  identify  the  impact  of  discrepancies  between  the  anticipated  and  real 
temperature  and  solar  irradiance  profiles.  Major  Newell  reran  the  model  with  the 
measured  temperature  and  solar  irradiance  profiles  for  that  period  of  time.  The 
results  of  these  new  HOMER  runs  showed  that  the  differences  between  real  world 
and  anticipated  temperature  and  solar  irradiance  profiles  accounted  for  a  10% 
overestimate:  a  6%  overestimate  in  power  production  was  due  to  differences 
between  real  and  anticipated  temperature  profiles,  while  a  4%  overestimate  in  PV 
power  production  was  due  to  differences  between  real  and  anticipated  solar 
irradiance  profiles  (Newell  2010).  This  still  leaves  a  large  gap  between  the  predicted 
and  actual  performance  of  the  PV  arrays. 

Assessing  the  Robustness  of  the  HOMER  Model 

HOMER  is  designed  to  identify  the  most  fuel-efficient  (i.e.,  cost-effective) 
solution  for  a  given  load  profile  at  a  fixed  location,  as  described  above.  The  answer 
is  a  single  “optimal”  configuration,  and  a  specified  fuel  usage/overall  cost.  We  now 
examine  several  extensions  that  seek  to  reveal  the  robustness  of  this  type  of 
solution. 
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Exploring  Spatial  Variability 

The  nature  of  Marine  Corps  expeditionary  operations  makes  the  effects  of 
irradiance  variance  even  more  important  than  in  the  commercial  settings  of  the  solar 
industry.  Due  to  the  unpredictable  nature  of  combat  and  humanitarian  operations, 
the  Marine  Corps  does  not  have  the  luxury  of  being  able  to  gather  data  on  the  solar 
radiance  and  weather  patterns  of  a  location  for  months  (or  years)  prior  to 
deployment.  Figure  3  shows  how  the  TMY3  data  varies  by  location  in  the  United 
States.  Clearly,  the  location  plays  a  vital  role  in  determining  the  utility  of  PV  arrays. 
Microclimates  also  have  an  effect.  For  example,  arrays  set  up  near  tall  trees,  tall 
buildings,  or  a  steep  hillside  produce  noticeably  less  power  than  arrays  set  up  a 
short  distance  away  without  obstructions. 


Figure  3.  Annual  solar  irradiance  in  the  United  States 

(from  USEIA,  2013). 

For  purposes  of  illustration,  we  restrict  our  investigation  to  60-day  operations 
taking  place  in  Salt  Lake  City,  Utah  beginning  on  the  75th  day  of  the  year.  For  a 
much  broader  investigation  of  ten  U.S.  cities,  other  operation  start  times,  and  other 
operation  duration  times,  see  Morse  (2014).  To  be  useful  for  deployment  to 
unanticipated  locations,  FIOMER  must  be  robust  enough  to  handle  the  solar 
irradiance  and  temperature  variances  of  those  locations.  Profiles  for  generic  regions 
(desert,  jungle,  mountain,  etc.)  are  desirable.  Such  profiles  will  be  associated  with 
greater  uncertainty  than  those  for  a  known  location,  but  are  beyond  the  scope  of  this 
paper. 
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Exploring  Temporal  Variability 

The  shorter  term  duration  of  expeditionary  operations  means  that  PV  systems 
used  by  the  Marine  Corps  are  also  quite  susceptible  to  the  effects  of  temporal  solar 
irradiance  variability.  This  is  not  surprising:  TMY3  data  is  intended  for  use  in 
simulating  solar  power  systems  over  “a  longer  period  of  time,  such  as  30  years" 
(Wilcox  and  William,  2008).  In  fact,  the  first  page  of  the  TMY3  User  Manual  warns  in 
italics:  “The  TMY  should  not  be  used  to  predict  weather  for  a  particular  period  of 
time"  (Wilcox  and  William,  2008).  The  subplots  in  Figure  4  make  this  temporal 
variation  apparent  on  a  monthly  basis,  but  still  portray  only  a  portion  of  the  true 
variability. 


Figure  4.  Monthly  direct  normal  irradiance  (DNI)  interannual  coefficient  of 

variation  (COV)  in  the  United  States 

(from  Gueymard  &  Wilcox,  201 1 ). 

As  an  example,  consider  two  graphs  that  show  the  variability  in  60  days  of 
solar  irradiation  for  Salt  Lake  City  (Figures  5  and  6),  beginning  on  the  75th  day  of  the 
year.  The  first  indicates  that  although  the  TMY  average  is  317  kilowatt-hours  per 
square  meter  (kWh/m^),  it  ranges  from  a  low  of  263  kWh/m^  to  a  high  of  359 
kWh/m^:  the  coefficient  of  variation  cr  //u  is  7.14%. 
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Figure  5.  Histogram  of  totai  soiar  irradiation  over  days  75-134 
for  Sait  Lake  City,  by  year,  1961-2010. 

A  closer  look  at  the  data  reveals  much  more.  Figure  6  plots  the  annual  solar 
irradiation  by  year.  The  line  represents  the  least-squares  regression  trend,  the  dark 
pink  region  represents  a  95%  confidence  interval  for  the  mean  response,  and  the 
light  pink  region  represents  a  95%  prediction  interval  for  an  individual  year's 
response.  The  trend  is  statistically  significant  (p-value  <  0.0001). 
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Figure  6.  Scatterplot  of  total  solar  irradiation  over  days  75-134  for  Salt  Lake 

City,  by  year,  1961-2010. 


Morse  (2014)  found  similar  patterns  for  the  other  nine  cities  he  studied.  After 
further  inquiry,  he  found  this  trend  was  a  manifestation  of  a  phenomenon  known  as 
"global  dimming"  (Muller,  2014).  Global  dimming  is  described  as  the  "decadal 
decrease  of  surface  solar  radiation"  (Wild,  2009).  The  name  "global  dimming"  was 
coined  by  Stanhill  &  Cohen  in  their  2001  article  “Global  Dimming:  A  Review  of  the 
Evidence  for  a  Widespread  and  Significant  Reduction  in  Global  Radiation”  (Stanhill 
&  Cohen,  2001 ).  The  “global”  portion  of  the  name  does  not  refer  to  the  phenomenon 
having  a  universal  effect  across  the  world  but  instead  refers  to  “the  sum  of  diffuse 
and  direct  solar  radiation,”  that  is  to  say  global  radiation  (Wild,  2009).  The  dimming 
refers  to  a  decrease  in  that  global  radiation  which  was  measured  as  occurring  at 

ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OE  BUSINESS  &  PUBLIC  POLICY  - 10  - 

NAVAL  POSTGRADUATE  SCHOOL 


different  rates  across  the  world.  In  the  1 980s  and  90s  climate  modelers  began  to 
distinguish  a  trend  of  decreasing  surface  solar  radiation  in  available  surface  solar 
radiation  records  dating  back  to  the  1 950s.  The  rates  of  the  decrease  differed  by 
location  but  were  mostly  seen  as  occurring  from  the  1950s  up  to  the  1980s  Muller, 
2014).  Later  studies,  focusing  on  the  period  from  the  1980s  to  2000s,  identified  an 
apparent  shift  in  the  trend  to  a  possible  increase  in  solar  irradiance.  This  trend  was 
named  “global  brightening.” 

The  performance  of  the  model  with  regard  to  long-term  solar  irradiance  trends 
is  important  because  of  its  effect  on  each  solar  power  system’s  performance  over  its 
life  cycle.  GREENS  performance  estimates  based  off  of  ten-year-old  TMY  data,  for 
example,  will  overestimate  the  solar  irradiance  available  to  power  the  system  where 
there  is  a  negative  linear  relationship  between  time  and  solar  irradiance.  Morse 
(2014)  provides  a  more  detailed  discussion  of  the  global  dimming  effects  as  they 
relate  to  the  use  of  HOMER. 

Exploring  the  Impact  of  Correlated  Costs 

We  have  already  discussed  several  of  the  inputs  to  the  HOMER  model, 
including  the  resources  available  (e.g.,  generators,  PV  arrays),  the  location  and 
associated  solar  and  temperature  information  (either  as  typical  years,  or  actual 
data),  load  profile  requirements,  operational  duration,  and  operational  start.  Other 
inputs  that  we  have  not  yet  discussed  include  cost  information — specifically,  costs  of 
capital,  replacement,  fuel,  and  operations  and  maintenance  (O&M)  costs  of  power 
components  (generators,  PV  arrays,  wind  turbines,  and  others  modeled  by 
HOMER).  The  user  specifies  these  costs  before  HOMER  is  executed.  Once  all  this 
information  is  set,  then  HOMER  will  output  the  configuration  it  deems  “optimal”  for 
the  specific  deterministic  setting,  and  provide  as  deterministic  outputs  the  expected 
costs  and  energy  usage  associated  with  each  platform.  Presumably,  this  is  the 
system  configuration  that  will  be  chosen  for  that  type  of  operation. 

The  default  inputs  for  HOMER  assume  that  the  relative  costs  of  conventional 
fuel  and  alternative  energy  sources  remain  constant.  However,  this  is  unlikely  to 
hold.  For  example,  one  could  argue  that  as  non-renewable  energy  sources  are 
depleted  and  become  scarcer,  they  will  become  more  expensive — and  that  the  costs 
for  renewable  energy  generators  will  tend  to  be  higher  as  well,  because  of 
competing  demands  for  limited  resources.  If  so,  this  assumes  there  will  be  a 
positive  correlation  between  the  two  types  of  costs  in  the  future.  Alternatively,  one 
could  argue  that  high  costs  for  non-renewable  sources  will  result  in  additional  R&D 
efforts  put  toward  renewable  energy  platforms,  so  there  will  be  a  tendency  for  these 
two  costs  to  be  negatively  correlated  in  the  future.  Indeed,  such  projections  are 
already  available.  For  example.  Figure  7(a)  (US  Energy  Information  Administration, 
2014)  shows  three  variants  of  oil  cost  projections  over  the  next  25  years:  a 
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reference  line,  as  well  as  low  and  high  oil  price  projections.  Figure  7(b)  (adapted 
from  US  Department  of  Energy,  2014)  shows  a  single  cost  projection  for  PV  arrays 
over  a  different  time  frame.  We  see  that  if  the  low  oil  price  projection  holds,  the  two 
energy  costs  will  be  positively  correlated  over  the  2014-2020  time  frame.  In  the 
other  two  circumstances,  the  two  energy  costs  will  be  negatively  correlated. 


Figure  7.  Oil  cost  projections  (from  USEIA,  2014)  and 
PV  array  cost  projections 

(adapted  from  USDOE,  2014). 


What  is  the  potential  impact  of  this  source  of  uncertainty  on  the  life  cycle  cost 
outputs  from  the  HOMER  model?  To  answer  this  question,  we  will  leverage  the  field 
of  design  of  experiments  (DOE)  in  order  to  quickly  calculate  the  results.  However, 
the  standard  DOE  approaches  are  not  suitable  in  the  presence  of  correlation.  We 
provide  technical  justification  in  the  next  section,  and  then  return  to  our  example. 

Methodological  Development:  Designs  for  Correlated  Noise  Factors 

Let  Y  denote  the  performance  measure  of  interest,  denote  the  noise 

factors  controlled  during  the  course  of  the  experiment,  and  let  //,  and  cry  denote  the 
mean  and  standard  deviation  of  W(i  =  Then,  assuming  that  the  noise 

factor  ranges  are  small  enough  that  a  linear  model  is  appropriate,  we  have 

r=  Po+PlW^  +  P2W2  +  ■■■  +  PkWk.  (1) 

An  orthogonal  experimental  design  (often  a  factorial  or  fractional  factorial)  is 
used  for  collecting  data  regarding  the  process.  Under  an  independence  assumption, 
setting  the  levels  for  noise  factor  W  at  /Uj  ±  o-j  will  result  in  a  two-point  distribution 
with  the  same  mean  and  standard  deviation  as  the  underlying  distribution.  After 
data  are  gathered,  a  linear  model  is  fit: 
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Y  =  p^  +  +  P2W2  +  -  + 


(2) 


Assuming  that  the  Ware  independent  and  normally  distributed,  then  the 
model  of  equation  (2)  can  be  used  to  determine  the  overall  system  mean  and 
variance  by  treating  the  estimated  coefficients  as  constants: 


(3) 


(4) 


From  equation  (4),  we  can  estimate  a  transmitted  variance  for  each  noise 
factor  i,  which  is  the  variability  in  the  response  that  is  attributable  to  Wr. 


(5) 


The  dependence  of  at,i  on  the  regression  coefficient  means  that  it  is  possible 
for  a  particular  noise  factor's  variance  to  be  magnified  or  dampened  as  it  is 
transmitted  through  to  the  response.  The  ratio  (at,,  /  a/)^  is  the  proportion  of  the 
variability  in  the  response  that  is  attributable  to  W,.  Potential  changes  to  the  noise 
factors,  that  affect  only  the  variance,  not  the  mean,  can  then  be  evaluated  by 
considering  the  reduction  on  the  overall  performance  variability  ay  and  the 
corresponding  reduction  in  cost. 

Now  suppose  that  two  of  the  /r noise  factors  ( Wi  and  W)  have  a  correlation  of 
p  in  the  real  world,  and  all  other  noise  factor  pairs  are  independent.  In  this  case,  the 
value  of  //r  remains  unchanged  from  that  computed  in  equation  (3).  However,  the 
overall  variance  now  is 


(6) 


which  is  greater  than  the  value  in  equation  (4)  if  p  >  0  and  less  than  this  value  if  p  < 
0.  This  means  that  the  overall  estimate  of  system  variability  and  the  transmitted 
variance  proportions  are  different  than  those  calculated  assuming  independence. 

If  the  Wi  (i  =  1,  are  truly  independent,  the  transmitted  variances 
completely  separate  the  effects  for  the  various  noise  factors.  This  does  not  occur 
when  Wi  and  W2  are  correlated,  although  orthogonal  experimental  designs  might  be 
desirable  because  of  the  increased  precision  of  the  estimated  coefficients. 

However,  the  results  are  no  longer  separable  in  the  real  world.  Rather  than  compute 
individual  transmitted  variances  for  each  of  the  two  factors,  we  use  a  joint 
transmitted  variance: 


^1,1,2  =  ^1(^1  +  ^2^2  +  ‘2pPip2(Tia2 
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Potential  changes  to  the  system  components  can  then  be  assessed  by 
determining  the  resulting  changes  in  the  joint  transmitted  variance  and  the  overall 
system  variance. 

This  approach  can  be  generalized  in  a  straightforward  manner  to  situations 
where  multiple  pairs  of  correlated  noise  factors  exist.  As  above,  we  construct  linear 
models  for  the  mean  and  variance  of  the  performance  measure  Y  as  functions  of  the 
noise  factors  Wj  {j= 


k 

^=l  (8) 

Treating  the  regression  coefficients  in  equation  (7)  as  constants,  we  obtain 

k  k  k 

i=l  i=l  j=i+l  (9) 

where  yOy  denotes  the  correlation  between  W,  andi  Wj.  Joint  transmitted 
variances  can  be  computed  as  in  equation  (6). 

In  some  situations,  the  correlation  in  the  real  world  may  restrict  the  choice  of 
the  design:  a  factorial  or  fractional  factorial  design  covering  the  entire  range  of 
interest  may  be  impossible  to  conduct.  For  example,  a  queueing  system  might  be 
unstable  if  all  noise  factors  are  held  at  their  high  levels.  If  this  situation  were  unlikely 
to  occur  in  practice  because  of  negative  correlation  among  the  variables,  then  a 
sampling  scheme  that  makes  use  of  the  underlying  dependence  structure  would 
seem  more  appropriate.  Alternatively,  it  may  be  that  the  statistical  response-surface 
models  are  developed  from  observational  rather  than  experimental  data.  Although 
such  models  can  be  analyzed  post  hoc  by  conducting  orthogonal  experiments 
involving  the  noise  factor  settings,  this  approach  is  questionable  if  the  model  was  not 
constructed  using  similar  combinations.  Large  extrapolations  of  the  fitted  model 
might  not  be  at  all  close  to  the  true  response  surface.  In  addition,  if  certain 
combinations  of  the  factors  are  not  likely  to  occur  in  practice,  then  we  lose  the 
benefit  of  sampling  at  levels  such  as  yu  ±  cr  in  order  to  achieve  the  same  variability  in 
the  sampling  distribution  as  in  the  underlying  normal  distribution. 

Once  again,  for  simplicity,  we  assume  that  the  noise  factors  are  normally  distributed 
but  only  the  first  two  {Wi  and  IV2)  are  correlated.  We  also  assume,  without  loss  of 
generality,  that  =  0  and  o-j  =  1  (/=  1,...,k).  This  corresponds  to  standardizing 
each  of  the  noise  factors  by  subtracting  its  mean  and  dividing  by  its  standard 
deviation  before  conducting  the  experiment.  The  joint  distribution  of  Wi  and  W2  is  then  a 
bivariate  normal.  Letting  Wi  be  on  the  x-axis  and  W2  on  the  y-axis,  the  contours  of  the 
bivariate  distribution  are  elliptical,  satisfying 
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—  2pxy  + 


(10) 


as  long  as  |p|  <  1  (Dudewicz  &  Mishra,  1987). 

If  yO=  0,  then  equation  (10)  reduces  to  a  circle  with  radius  r  centered  about 
the  origin.  If  /?  >  0,  then  the  major  axis  of  the  ellipse  falls  on  the  line  y- x  and  the 
minor  axis  is  along  the  line  y-  -x,  as  shown  in  Figure  8.  If  we  ran  our  experiment  as 
if  Wi  and  Wi  were  independent,  then  a  2-level  factorial  would  result  in  sampling  at 
the  4  points  (±1,  ±1).  These  correspond  to  sampling  each  of  the  two  factors  at 
Pi  ±(Ti,  SO  that  the  discrete  sampling  distributions  for  Wi  and  Wi  have  the  same 
mean  and  variance  as  the  marginal  distributions.  An  alternative,  which  looks  much 
more  reasonable  in  the  presence  of  correlation,  is  to  sample  at  points  on  one  of  the 
elliptical  contours. 


W 


Figure  8.  Density  contours  for  a  bivariate  standard  normai  with 

correiation  coefficient  p  =  0.9. 


From  equation  (10)  and  basic  geometry  (Beyer,  1987)  one  can  compute 
coordinates  of  the  major  and  minor  axial  points  for  given  r.  Equations  (11)  and  (12) 
hold  when  p  >  0,  when  standardized  units  are  used  for  both  noise  factor  designs.  If 
p  <  0  then  equation  (1 1 )  gives  the  minor  axial  points  and  equation  (1 2)  gives  the 
major  axial  points. 

Major  Axial  Points 


(11) 


Minor  Axial  Points 


+ 


(12) 


V2(1+p)’V2(1+P)/ 
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If  we  sample  at  the  four  axial  points,  then  the  variance  of  the  sampling  distribution 
for  either  Wi  or  Wi  will  be 


2  _  1  /"  I  ^ 


(13) 


_  f  1+p+l-p 
~  4  V(l+p)(l-p)/  “  2(l-p2) 

In  order  to  achieve  appropriate  marginal  variances,  set  equation  (13)  equal  to  the 
marginal  variance  (oi^  =  a2^  =  1)  and  solve  for  r.  This  yields  /  =  so  the 

experimental  design  will  consist  of  sampling  at  the  major  and  minor  axial  points  of 
the  ellipse  specified  by 


—  2pxy  +  =  2(1  —  p^) 


(14) 


A  plot  of  such  ellipses  is  shown  in  Figure  9,  with  correlations  ranging  from 
zero  (a  circle)  to  0.9.  For  negative  correlations,  the  mirror  image  of  this  figure  is 
appropriate.  The  standard  two-level  factorial  design  (with  sampling  of  both  factors  at 
one  standard  deviation  below  and  one  standard  deviation  above  the  mean)  is  the 
limiting  case  as  p  approaches  zero.  Flowever,  the  design  is  quite  different  if 
correlation  is  present:  the  individual  factor  ranges  are  larger  than  Pi  ±  cTj,  but  the 
two-dimensional  region  over  which  sampling  takes  place  is  narrower. 


Figure  9.  Elliptical  design  points  for  two-level  sampling  under  correlation, 

for  p  =  0  (circle),  0.1 , 0.3,  0.5,  0.7,  and  0.9. 

These  elliptical  designs  were  introduced  in  Sanchez  (1994b),  where  they 
were  illustrated  for  two  diverse  applications:  a  deterministic  circuit  model,  and  a 
stochastic  home  mortgage  portfolio  model. 
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For  a  deterministic  model,  such  as  the  HOMER  example  used  in  this  paper, 
the  transmitted  variances  for  the  noise  factors  sum  to  the  overall  system  variance. 

In  discrete-event  simulation  experiments,  this  is  generally  not  the  case:  although 
many  noise  factors  can  be  explicitly  controlled,  the  system  is  characterized  by  some 
inherent  variability  that  remains  unexplained  by  any  noise  factors.  Tolerance  design 
analysis  must  therefore  be  modified  to  take  this  into  account  (Sanchez,  1994a; 
Sanchez  et  al.  1 998).  In  order  to  reflect  the  true  system  variability,  the  overall 
system  variance  estimate  of  equation  (4)  is  augmented  by  incorporating  estimates  of 
the  system  variability  obtained  within  simulation  runs.  This  augmented  analysis 
should  be  used  for  either  orthogonal  or  elliptical  tolerance  experiment  designs  so  the 
results  reflect  the  true  system  variability. 

Correlated  Costs  for  the  HOMER  Example 

We  now  illustrate  the  approaches  for  the  HOMER  example.  We  simplify  the 
setting  for  illustration  purposes.  Our  assumptions  follow. 

•  Let  /"denote  the  number  of  liters  of  diesel  fuel  used. 

•  We  are  concerned  with  two  noise  factors:  IV^  is  the  cost  of  diesel  fuel 

($/liter),  and  I/Kp^  is  the  purchase  cost  of  PV  arrays  ($/kW  peak  load 
requirement).  We  assume  that  these  noise  factors  are  normally 
distributed:  Wa-N(l,0.37^)  and  (360,66.6^). 

•  Our  peak  load  requirement  for  the  5  GREENS  system  is  1  kW,  and  the 
liters  of  fuel  used  are  determined  from  HOMER. 

•  We  remove  the  linear  global  dimming  trend  from  the  data,  and  treat  the 
variation  around  this  trend  line  (due  to  variation  in  solar  irradiation  and 
temperature)  as  the  intrinsic  variability  in  the  system. 

•  Our  response  Y  is  computed  by  considering  the  peak  requirement  for 
the  5  GREENS  system  (np^  =5)  and  the  liters  of  diesel  fuel  saved  (n^): 

V  =  Up^Wp^  +  FWa. 

This  is  the  energy  cost  if  the  PV  arrays  are  used  for  a  single  operation. 
A  more  realistic  assumption  is  that  they  will  be  used  for  multiple 
operations,  in  which  case  the  cost  of  the  arrays  would  be  prorated 
accordingly.  In  our  simple  example,  we  use  V  to  compare  the  relative 
merits  of  different  alternatives. 

•  We  ignore  all  other  costs  and  discount  factors. 

Consider  three  different  correlations  for  the  noise  factors  and  Wpj,:  p  -  0 
(no  correlation),  p  =  0.9  (strong  positive  correlation),  and  p  =  -0.9  (strong  negative 
correlation).  Figure  10  illustrates  the  differences  between  several  of  the  estimates 
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that  we  have  discussed.  In  Figure  1 0(a)  we  plot  (i)  the  deterministic  answer  from 
HOMER,  using  TMY  inputs  for  both  solar  and  temperature,  and  the  expected  noise 
factor  costs;  (ii)  the  distribution  one  might  use  based  on  the  recorded  solar 
irradiation  and  temperature  variability  if  one  assumed  the  data  from  different  years 
were  independent  and  identically  distributed;  and  (iii)  the  predicted  distribution  based 
on  the  different  solar  irradiation  and  temperature  variability — this  removes  the  global 
dimming  trend,  and  represents  the  intrinsic  variability  in  the  system  (at  least  for 
short-term  projections).  This  clearly  illustrates  the  importance  of  exploring  the 
uncertainty  in  the  potential  operational  environmental  conditions,  in  order  to  convey 
the  resulting  risk  to  the  decision  maker.  For  example,  while  Marine  Corps  officers 
might  not  be  interested  in  estimating  cost  when  planning  an  expeditionary  operation, 
the  amount  of  conventional  fuel  saved  might  directly  affect  the  length  of  time  they 
can  sustain  this  operation  before  resupply. 

In  Figure  10(b)  we  include  uncertainties  in  the  noise  factors  (future  PV  array 
cost  and  diesel  fuel  cost)  to  the  one-year  projection  that  incorporates  global 
dimming.  Note  the  vastly  different  scales  in  Figures  10(a)  and  10(b).  This  shows 
how  uncertainties  in  future  costs — and  the  correlations  in  these  uncertainties — 
translate  to  probabilities  of  cost  overruns.  This  may  be  of  interest  to  program 
managers  as  they  seek  to  acquire  new  systems.  In  our  example,  all  three  of  the 
correlation  assumptions  yield  distributions  with  the  same  mean  ($5189),  but  the 
standard  devations  (a's)  differ.  The  smallest  a  occurs  when  the  future  costs  of  PV 
arrays  and  diesel  fuel  are  negatively  correlated,  and  the  largest  a  occurs  when  they 
are  positively  correlated. 
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(a)  Cost  distributions  based  on  different  assumptions  (b)  Cost  distributions  based  on  global  dimming,  with 

regarding  uncertainties  in  soiar  and  temperature  data  different  assumptions  regarding  correiations  in  future 

costs  of  PV  arrays  and  diesei  fuei 

Figure  10.  Distributions  of  K (combined  cost)  based  on  different 
assumptions  for  uncertainties  in  future  soiar,  temperature,  and  cost 

component  distributions. 

The  different  distributions  translate  to  different  risks  of  exceeding  particular 
cost  thresholds.  Numerical  comparisons  are  easier  to  show  using  cumulative 
distribution  plots,  as  in  Figure  1 1 ,  instead  of  the  density  plots  in  Figure  10(b).  Figure 
1 1  shows  that  a  program  manager  interested  in  estimating  the  probability  of 
exceeding  a  cost  threshold  of  $7000  can  use  the  appropriate  curve.  This  probability 
is  0.894  (89.4%)  when  p  =  0.5,  0.918  when  p  =  0,  and  0.958  when  p  =  -0.7. 
(Alternatively,  if  one  were  interested  in  the  cost  threshold  that  would  be  exceeded 
with  probability  at  most  0.10,  this  translates  to  cutoffs  of  $7047,  $6833,  and  $6533, 
for  p  -  0.5,0,  and  -0.7,  respectively.  In  some  instances,  differences  in  risk  might  be 
enough  to  change  a  decision  to  continue  with  a  program,  and  care  should  be  taken 
to  select  appropriate  noise  factor  distributions  and  correlations.  In  our  example,  the 
striking  differences  between  estimates  in  Figures  10(a)  and  10(b)  indicate  that 
including  the  noise  factor  distributions  at  even  a  rough  level  is  extremely  informative. 
Ignoring  these  (Figure  10(a))  would  lead  a  program  manager  to  assume  there  was 
essentially  no  risk  of  cost  exceeding  $5400,  when  the  Figure  10(b)  and  Figure  (11) 
show  that  the  real  risks  would  be  over  44%. 

These  simple  numerical  examples  are  for  illustration  purposes  only.  To  truly 
be  useful  for  program  managers,  life  cycle  cost  estimates  (rather  than  one-year-out 
projections) — and  a  more  comprehensive  examination  of  the  breadth  of  operational 
requirements  and  operational  environments — are  necessary.  Furthermore,  robust 
design  principles  can  be  used  to  seek  system  configurations  that  are  robust  to 
uncertainties  in  the  environment,  which  is  an  important  consideration  for  equipping 
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expeditionary  units.  Once  again,  we  refer  the  reader  to  Morse  (2014),  who 
conducted  a  much  larger  experiment  over  operations  of  start  times,  durations,  and 
locations  for  units  with  different  configurations  of  energy  supply  resources. 


Figure  11.  CDFs  of  /(combined  cost)  based  on  different  assumptions  for 
cost  component  correiations,  using  giobai  dimming  resuits  for  soiar 
and  temperature  uncertainties. 

Concluding  Remarks 

In  this  paper,  we  provided  motivation  for  the  need  to  develop  methods  for 
conducting  designed  experiments  involving  correlated  noise  factors.  We  show  how 
these  can  be  used  to  obtain  a  more  representative  distribution  of  potential  life  cycle 
costs,  so  that  program  managers  and  planners  can  have  better  assessments  of  the 
potential  risks  associated  with  acquiring  or  employing  particular  systems.  These 
robust  assessments  are  important.  By  revealing  the  risk  of  budget  overruns 
associated  with  going  forward  with  a  particular  system,  they  provide  the  decision 
makers  with  much  more  information  than  a  simple  “yes  or  no”  comparison  to  a  fixed 
budget  constraint.  At  the  same  time,  a  designed  experiment  approach  can  be  much 
more  efficient  than  Monte  Carlo  simulation.  We  are  currently  evaluating  other  types 
of  space-filling  designs  that  may  be  more  suitable  for  situations  where  the 
uncertainties  may  be  far  from  normally  distributed. 

From  the  applications  perspective,  this  example  in  this  paper  is  intended  as  a 
simple  illustration  of  the  need  for  robust  cost  estimation,  rather  than  a  definitive 
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estimate  for  solar  PV  arrays.  Other  uncertainties  in  the  environmental  conditions, 
such  as  temperature  variations,  will  also  play  a  role.  Many  other  components 
influence  life  cycle  costs — O&M  cost,  cost  of  spare  parts,  and  discount  rates,  to 
name  a  few.  There  are  also  intangibles  that  may  be  more  difficult  to  quantify,  but 
that  play  major  roles  in  determining  the  value  of  these  systems.  For  example,  the 
ability  to  partially  offset  energy  needs  with  PV  arrays  may  increase  the  time  an 
expeditionary  unit  conducting  disaster  relief  efforts  is  self-sustainable:  this,  in  turn, 
may  influence  its  relief  effort  plans.  Yet  our  preliminary  work  is  promising.  We  hope 
the  solar  energy  example  in  this  paper  motivates  the  need  for  obtaining  robust  cost 
estimates,  and  shows  how  robust  design  principles  can  assist  in  this  estimation 
process. 
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